X Reality Networked Performance: Message Based Distributed Systems for Controlling and Presenting Multiple Realities

X reality networked performances connect physical, fictional and computer generated realities in a new world of performance, one that is without geographical bounds and that can include many physical locations—with their own performers and audience members— within a single event. They explore a unique medium while drawing on historical and contemporary performing arts practices that normally occur within the confines of a single physical location. Such performances present a special set of requirements on the system that supports them. They need to access and integrate all the systems that are typically found in the physical place of the performance (such as theatre lighting) with those that are unique to the medium, such as network technologies and environments for the delivery of computer generated realities. Yet, no suitable systems or frameworks have been developed to support them. Technologies are available (such as LoLA and UltraGrid) that support individual aspects—like audio/video streaming—but which do not address the wider requirements of controlling and synchronising, of integrating all these technologies into a system of systems for X reality networked performance. Therefore, this research investigates the creation of a systems framework whereby existing hardware and software components can be continuously integrated with bespoke components to provide a platform for the delivery of X reality networked performances. The methodological approach to this investigation is through the lens of the author’s previous experience in other fields of complex systems integration, including, approaches employed in the design and integration of avionics systems. Specifically, it tests if a systems integration approach to providing a technical platform for X reality networked performances, one that employs strongly‐defined interfaces and communication protocols, and that is based on open and industry standards, delivers an elegant platform that can be characterised as: deterministic, reliable, extendable, scalable, reconfigurable, testable and cost effective. The platform for X reality networked performance has been developed iteratively—using the results of a framework investigation—and tested in four different performance projects over a period of 24‐months, in ten different venues, across five countries. The research concludes that the enabling framework is well suited to the delivery of X reality networked performances. Also, that the approaches employed could equally be applied to the needs of other arts practitioners who rely on complex technical systems for the creation and delivery of their work

X Reality Networked Performance: Message Based Distributed Systems For Controlling And Presenting Multiple Realities

X reality networked performances connect physical, fictional and computer generated realities in a new world of performance, one that is without geographical bounds and that can include many physical locations—with their own performers and audience members— within a single event. They explore a unique medium while drawing on historical and contemporary performing arts practices that normally occur within the confines of a single physical location. Such performances present a special set of requirements on the system that supports them. They need to access and integrate all the systems that are typically found in the physical place of the performance (such as theatre lighting) with those that are unique to the medium, such as network technologies and environments for the delivery of computer generated realities. Yet, no suitable systems or frameworks have been developed to support them. Technologies are available (such as LoLA and UltraGrid) that support individual aspects—like audio/video streaming—but which do not address the wider requirements of controlling and synchronising, of integrating all these technologies into a system of systems for X reality networked performance. Therefore, this research investigates the creation of a systems framework whereby existing hardware and software components can be continuously integrated with bespoke components to provide a platform for the delivery of X reality networked performances. The methodological approach to this investigation is through the lens of the author’s previous experience in other fields of complex systems integration, including, approaches employed in the design and integration of avionics systems. Specifically, it tests if a systems integration approach to providing a technical platform for X reality networked performances, one that employs strongly‐defined interfaces and communication protocols, and that is based on open and industry standards, delivers an elegant platform that can be characterised as: deterministic, reliable, extendable, scalable, reconfigurable, testable and cost effective. iii The platform for X reality networked performance has been developed iteratively—using the results of a framework investigation—and tested in four different performance projects over a period of 24‐months, in ten different venues, across five countries. The research concludes that the enabling framework is well suited to the delivery of X reality networked performances. Also, that the approaches employed could equally be applied to the needs of other arts practitioners who rely on complex technical systems for the creation and delivery of their work

Necessidades dos usuários de espaços de performances imersivas mediatizadas

Over the last two decades performance artists have made use of videoconferencing systems for installations, joint performances and rehearsals; often negotiating costly and bespoke (custom-made) soft and hardware. More recently, pervasive and low-cost videoconferencing technology has successfully been applied in mediated installations, paving the way for the technology to be incorporated into the curriculum of performance colleges thereby creating a need for systematic user requirement research. We report on a study in which theatre and dance students collaborated across a videoconferencing connection. The results highlighted different requirements for distributed theatre and dance practitioners and these findings form an input into an EU Framework 7 research program focusing on how to better support the mediated performance space and enable audience feedback and interaction. We highlight theoretical perspectives relevant to telepresence in performance. Lastly we describe how insights into user requirements for immersive mediated performance spaces led to building a prototype consisting of two Computer Aided Virtual Environments (CAVEs) connected via the Internet.Nas duas últimas décadas artistas performáticos vêm fazendo uso desistemas de videoconferências para instalações, performances conjuntas e ensaios, frequentemente negociando por preço elevado tanto o software quanto o hardware “bespoke” (feito sob encomenda). Recentemente, tecnologia pervasiva de baixo-custo para videoconferência tem sido aplicada em instalações mediadas, abrindo o caminho para uma tecnologia a ser incorporada no currículo das escolas superiores de performance, criando assim a obrigação de uma investigação sistemática sobre as necessidades do usuário. Relatamos um estudo no qual estudantes de teatro e de dança compartilham experiência por meio de conexão de videoconferência. Os resultados ressaltam as diferentes exigências feitas aos difusores de práticas de teatro e dança e esses resultados contribuem para um programa de investigação da UE Framework 7, cujo enfoque é a melhor forma de apoiar o espaço de atuação mediada, e permitir resposta interativa do público. Ressaltamos perspectivas teóricas relevantes para a telepresença em performance. Por último, descrevemos como insights sobre as necessidades dos usuários de espaços de performances imersivas mediadas levaram à construção de um protótipo que consiste em duas cavernas (CAVEs – Ambientes virtuais em Computer Aided [CAD]), conectadas através da internet

Getting Closer – The Digital Tempest

Presentation to the Co-Creation, distributed performances and alternative content for the big screen workshop, Barcelona, Institue del Teatre. 10 – 11 November 2014

How Are We Connected? Measuring Audience Galvanic Skin Response of Connected Performances."

Accurately measuring the audience response during a performance is a difficult task. This is particularly the\ud case for connected performances. In this paper, we staged a connected performance in which a remote\ud audience enjoyed the performance in real-time. Both objective (galvanic skin response and behaviours) and\ud subjective (interviews) responses from the live and remote audience members were recorded. To capture\ud galvanic skin response, a group of self-built sensors was used to record the electrical conductance of the skin.\ud The results of the measurements showed that both the live and the remote audience members had a similar\ud response to the connected performance even though more vivid artistic artefacts had a stronger effect on the\ud live audience. Some technical issues also influenced the experience of the remote audience. In conclusion we\ud found that the remoteness had little influence on the connected performance

Designing a system for Online Orchestra: Computer hardware and software

Online Orchestra sought in its pilot performance to enable musicians in four remote locations in Cornwall, United Kingdom, to make music together over the Internet. This article considers the processes by which computer hardware and software platforms were selected, integrated and optimized for the performance. Starting with an overview of guiding design principles, details of the computer hardware and software platforms used are provided. Audio- and video-streaming solutions are surveyed, leading to a detailed analysis of JackTrip and its deployment in the final system. The network environment in which the Online Orchestra performance took place is also considered, including specifications of each of the four venues

Sensing Audience Response- Beyond One Way Streaming of Live Performances

During a live theatre performance, Galvanic Skin Response (GSR) of 15 audience members was measured simultaneously and synchronized with video footage of performers and audience. Questionnaires explored emotions evoked during the play. The research is part of an EU Framework 7 program to support remote and interactive performances

How Are We Connected? Measuring Audience Galvanic Skin Response of Connected Performances."

Accurately measuring the audience response during a performance is a difficult task. This is particularly the case for connected performances. In this paper, we staged a connected performance in which a remote audience enjoyed the performance in real-time. Both objective (galvanic skin response and behaviours) and subjective (interviews) responses from the live and remote audience members were recorded. To capture galvanic skin response, a group of self-built sensors was used to record the electrical conductance of the skin. The results of the measurements showed that both the live and the remote audience members had a similar response to the connected performance even though more vivid artistic artefacts had a stronger effect on the live audience. Some technical issues also influenced the experience of the remote audience. In conclusion we found that the remoteness had little influence on the connected performance

User requirements in immersive mediated performance spaces

Vconect (http://www.vconect-project.eu/) is an EU Framework 7 research program. One of the aims is to support the area of mediated performance for the purpose of remote teaching, rehearsals and performances through videoconferencing technologies featuring automated editing and composition of multiple live audio-video streams onto multiple screens. The orchestration of the video and audio streams will take into account a variety of editing cues originating from directional audio, face detection and physiological sensors augmenting audience feedback. \ud \ud Over the last two decades performance artists have made use of videoconferencing systems for installations, joint performances and rehearsals; often negotiating costly and bespoke soft and hardware. More recently, pervasive and low-cost videoconferencing technology has successfully been applied in mediated installations, paving the way for the technology to be incorporated into the curriculum of performance colleges thereby creating a need for systematic user requirement research. We report on a series of studies in which theatre and dance students collaborated across a videoconferencing connection. The results highlighted different technology requirements for distributed theatre and dance practitioners and these findings help shape the the Vonect research agenda. We highlight theoretical perspectives relevant to telepresence in performance

Online Orchestra Scalability Study

The Online Orchestra Scalability Study aimed to establish the scalability of Online Orchestra, and provide an evidence base that would help inform future investment. In collaboration with the Bournemouth Symphony Orchestra (BSO), we co-designed a workshop, delivered online on 10th November 2016, to test the feasibility of the BSO to augment its music education programme by delivering content online, thereby increasing its geographical reach to more distant locations. The workshop involved BSO musicians, located at the BSO headquarters in Poole, delivering an online workshop to a string quartet of four pupils, located in Jewell Academy, Bournemouth. The workshop was facilitated by Online Orchestra technology, and involved two parts. In part one, the string quartet performed an extract from Mozart’s Quartet in D minor, K421; BSO musicians then provided a 20-minute coaching session on the performance, in which they worked with the children to improve their performance. In part two, all musicians performed together, led by the Online Orchestra conductor. Musicians performed a specially-commissioned work written by Jim Aitchison (Variations on 3 Modes); this performance was rehearsed by the conductor over a 20-minute period